Introduce a process-wide singleton engine for .collect(engine="gpu")#22410
Merged
Conversation
madsbk
added
improvement
github-actions
Bot
added
Python
madsbk
force-pushed
the
default_gpu_engine
branch
9 times, most recently
from
7e6beeb to
0fb9fe8
Compare
madsbk
commented
May 9, 2026
| Because each call forks a new child, process-wide side-effects | ||
| (the ``_bind_done`` flag, CPU affinity, environment variables) never | ||
| leak between tests or back into the pytest process. | ||
| def _run_in_subprocess(target: Callable[[], None]) -> None: |
Member
Author
There was a problem hiding this comment.
This PR exposed some issues with the "fork" approach, so we now use "spawn" instead. Otherwise, the tests remain the same.
madsbk
commented
May 9, 2026
madsbk
commented
May 9, 2026
madsbk
added
breaking
wence-
reviewed
May 11, 2026
Co-authored-by: Lawrence Mitchell <wence@gmx.li>
Contributor
TomAugspurger
left a comment
TomAugspurger
left a comment
There was a problem hiding this comment.
Partial review. I'll try to get back to this later, but don't wait for me.
| streaming runtime. | ||
| * ``Cluster.DASK`` : Multi-GPU execution via Dask workers and the rapidsmpf | ||
| streaming runtime. | ||
| * ``Cluster.DEFAULT_SINGLETON`` : Single-GPU execution via the DefaultSingletonEngine. |
Contributor
There was a problem hiding this comment.
Let's confirm this is the name we want.
- if we ever change the default, then this name will become misleading
- How commonly understood is "singleton"?
- This name doesn't mention "single" GPU-only, though the docs do.
And given that this is the default... maybe we can get away with updating the call sites to cluster: Cluster | None? And if we encounter None then we set up the default singleton cluster, and so we don't even need an enum name for this thing?
Member
Author
There was a problem hiding this comment.
I guess you are right, but I still prefer the more descriptive name. I like DEFAULT_SINGLETON and DefaultSingletonEngine because that is exactly what they are :)
If we ever decide to change the default implementation, I think we should change what DefaultSingletonEngine does internally rather than route the default path to an entirely different engine type.
To me, the important semantic is “process-wide implicit singleton default engine”, and the current name makes that very explicit.
wence-
approved these changes
May 12, 2026
…/default_singleton_engine.py Co-authored-by: Lawrence Mitchell <wence@gmx.li>
Member
Author
|
/merge |
3 tasks
TomAugspurger
pushed a commit
to TomAugspurger/pygdf
that referenced
this pull request
May 12, 2026
rapidsai#22410) `lf.collect(engine="gpu")` and `pl.GPUEngine(executor="streaming")` using the default cluster now route through a new process-wide `DefaultSingletonEngine` instead of constructing a fresh rapidsmpf `Context`, RMM adaptor, and Python executor for every query. Bootstrap now happens once per process rather than once per query. `DefaultSingletonEngine` is a process-wide single-GPU singleton specialization of `SPMDEngine`: at most one live instance exists per process, it always uses a single-rank communicator plus default environment-derived settings, and repeated calls reuse the same engine instance until explicit shutdown. The default cluster enum value is renamed from `Cluster.SINGLE` to `Cluster.DEFAULT_SINGLETON` so the dispatch token better reflects the actual behavior. This PR also removes the dead inline-context fallback in `evaluate_pipeline`, which was the original `"single"` execution path. Authors: - Mads R. B. Kristensen (https://github.com/madsbk) Approvers: - Lawrence Mitchell (https://github.com/wence-) URL: rapidsai#22410
shrshi
pushed a commit
to shrshi/cudf
that referenced
this pull request
May 12, 2026
rapidsai#22410) `lf.collect(engine="gpu")` and `pl.GPUEngine(executor="streaming")` using the default cluster now route through a new process-wide `DefaultSingletonEngine` instead of constructing a fresh rapidsmpf `Context`, RMM adaptor, and Python executor for every query. Bootstrap now happens once per process rather than once per query. `DefaultSingletonEngine` is a process-wide single-GPU singleton specialization of `SPMDEngine`: at most one live instance exists per process, it always uses a single-rank communicator plus default environment-derived settings, and repeated calls reuse the same engine instance until explicit shutdown. The default cluster enum value is renamed from `Cluster.SINGLE` to `Cluster.DEFAULT_SINGLETON` so the dispatch token better reflects the actual behavior. This PR also removes the dead inline-context fallback in `evaluate_pipeline`, which was the original `"single"` execution path. Authors: - Mads R. B. Kristensen (https://github.com/madsbk) Approvers: - Lawrence Mitchell (https://github.com/wence-) URL: rapidsai#22410
galipremsagar
pushed a commit
to galipremsagar/cudf
that referenced
this pull request
May 13, 2026
rapidsai#22410) `lf.collect(engine="gpu")` and `pl.GPUEngine(executor="streaming")` using the default cluster now route through a new process-wide `DefaultSingletonEngine` instead of constructing a fresh rapidsmpf `Context`, RMM adaptor, and Python executor for every query. Bootstrap now happens once per process rather than once per query. `DefaultSingletonEngine` is a process-wide single-GPU singleton specialization of `SPMDEngine`: at most one live instance exists per process, it always uses a single-rank communicator plus default environment-derived settings, and repeated calls reuse the same engine instance until explicit shutdown. The default cluster enum value is renamed from `Cluster.SINGLE` to `Cluster.DEFAULT_SINGLETON` so the dispatch token better reflects the actual behavior. This PR also removes the dead inline-context fallback in `evaluate_pipeline`, which was the original `"single"` execution path. Authors: - Mads R. B. Kristensen (https://github.com/madsbk) Approvers: - Lawrence Mitchell (https://github.com/wence-) URL: rapidsai#22410
This was referenced May 13, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
lf.collect(engine="gpu")andpl.GPUEngine(executor="streaming")using the default cluster now route through a new process-wideDefaultSingletonEngineinstead of constructing a fresh rapidsmpfContext, RMM adaptor, and Python executor for every query. Bootstrap now happens once per process rather than once per query.DefaultSingletonEngineis a process-wide single-GPU singleton specialization ofSPMDEngine: at most one live instance exists per process, it always uses a single-rank communicator plus default environment-derived settings, and repeated calls reuse the same engine instance until explicit shutdown.The default cluster enum value is renamed from
Cluster.SINGLEtoCluster.DEFAULT_SINGLETONso the dispatch token better reflects the actual behavior.This PR also removes the dead inline-context fallback in
evaluate_pipeline, which was the original"single"execution path.